Semi-supervised learning (SSL) provides a powerful framework for leveraging unlabeled data when labels are limited or expensive to obtain. SSL algorithms based on deep neural networks have recently proven successful on standard benchmark tasks. However, we argue that these benchmarks fail to address many issues that SSL algorithms would face in real-world applications. After creating a unified reimplementation of various widely-used SSL techniques, we test them in a suite of experiments designed to address these issues. We find that the performance of simple baselines which do not use unlabeled data is often underreported, SSL methods differ in sensitivity to the amount of labeled and unlabeled data, and performance can degrade substantially when the unlabeled dataset contains out-of-distribution examples. To help guide SSL research towards real-world applicability, we make our unified reimplemention and evaluation platform publicly available.

This begs the question: Can SSL algorithms simultaneously improve upon both UL and SL?

It has been repeatedly shown that deep neural networks (DNNs) can achieve human-or superhuman-levelperformances on various tasks [1,17,32].

We respond to each comment one by one. We mention this in Line 148; however, we will make it clear in the final draft. Conversely, SSL algorithms use the unlabeled data but they do not consider the class imbalance. We will make this point clear in the final draft. However, to avoid the confusion, we will substitute X,Y to α,β in the final draft.

Semi-supervised learning (SSL) improves model generalization by leveraging massiveunlabeled datatoaugment limited labeled samples.